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BACKGROUND OF THE INVENTION 

Cross-Reference to Related Application 

This application claims priority under 35 U.S.C. 1 19(e) from U.S. provisional 
patent application 60/297,817, having a filing date of 06/1 1/2001 which is entitled 
"Handheld Device " inventors Tom Bridgwater, Debbie Chyi, Robert Haitani, Jeff 
5 Hawkins, Eric Jacobsen, Will Rees, Peter Skillman, and Karl Townsend and which is 
hereby incorporated by reference. 
Field of the Invention 

The present invention relates generally to systems and methods for symbol 
processing by a computing device, in particular a hand-held computing device. 
L= 10 Description of the Related Art 

p The integration of services into computer devices, including hand-held devices 

Q such as personal digital assistants, often involves an increased cost of time to the user to 

n 

adapt to a new man-machine interface. The man-machine interface aspect of the data 
entry mechanisms is crucial to decreasing the learning curve time customers need to fully 
1 5 utilize the device for performing its various functions in order to decrease customer 
frustration. The ease of manipulation of data entry mechanisms enhances the overall 
device experience. 

It is desirable to strike an appropriate balance between burdening the user with a 
bulkier device on the one hand with dedicated data entry mechanisms as part of the 
20 physical interface and on the other hand, providing a more limited man-machine interface 
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requiring the user to have to learn more data entry techniques or to consume more time 
for data entry. 

One example of how a limited man-machine interface can be time consuming 
occurs when a user desires to enter a character that is not located as part of the physical 
interface such as a keyboard or an initial view of a display presented to a user for entering 
text. For example, in trying to find an accented "e" version of the letter "e", typically the 
user must go to a separate display of a table of all accented characters and search through 
one or more displays of accented characters carefully so as not to miss the one for which 
he is looking. Otherwise, the user can memorize special key sequences. A more pleasant 
and robust text entry experience that gives the user easy access to a wide range of 
symbols is highly desirable. 

The man-machine interface impacts the acceptability of the device to users because it 
impacts the extent of services provided, the effective use of the services, and the overall 
reliability of the device. 

SUMMARY OF INVENTION 

The present invention provides for a system and method for processing alternate 
symbols associated with or linked to a base symbol in a computer device. Example of 
computer devices that may embody the system or method are hand-held computing 
devices. 

A base symbol is a symbol to which at least one alternate symbol is linked. A base 
symbol commonly appears on a key of a keyboard, a display of a keyboard, or is a 
handwritten symbol recognized by handwriting entry software. Examples of alternate 



2 



21495/05943/DOCS/1214427.5 



Fenwick & West 5943 PATENT 

symbols are accented characters and punctuation marks that do not appear on a keyboard 
or are not commonly recognized by a handwriting entry program, and short symbol 
sequences. An example of a common short symbol sequence is an emoticon used in e- 
mail messages to convey tone or feelings. 
5 In one embodiment of a system for performing the invention, the system comprises 

a display, a memory, a processor coupled to the display and memory, and software for 
processing alternate symbols which may be stored in the memory, an example of a 
computer usable medium, and executed by the processor. 

One embodiment of a computer-implemented method in accordance with the 
10 invention comprises receiving input indicating a request for an alternate symbol 
associated with abase symbol and displaying a list of at least one alternate symbol 
associated with the base symbol. 

As a base symbol may have more than one alternate symbol associated with it, a 
base symbol may have a list of alternate symbols associated with it. In one embodiment, 
15 the list of alternate symbols and information associated with the list are stored in a 

memory. However, the list and / or information associated with the list may be updated 
in the memory according to criteria. In one embodiment, the criteria may be based on 
prior usage, and in another, the criteria may be user-defined. 

20 BRIEF DESCRIPTION OF THE FIGURES 

The figures depict one or more embodiments of the present invention for purposes 
of illustration only. One skilled in the art will readily recognize from the following 
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discussion that alternative embodiments of the structures and methods illustrated herein 
may be employed without departing from the principles of the invention. 

Figure 1 A illustrates a front perspective view of an embodiment of a hand-held 
computing device with a lid opened in an extended position in accordance with the 
5 present invention. 

Figure IB illustrates a back perspective view of an embodiment of a hand-held 
computing device with a lid opened in an extended position in accordance with the 
present invention. 

Figure 1C illustrates a view of the front side of the embodiment of the hand-held 
10 device of Figure 1A in a lid closed state in accordance with the present invention. 

Figure 2 illustrates a block diagram of an embodiment of a system enclosed 
within the case of the hand-held device comprising hardware and software for performing 
functions including functions related to communication services and symbol processing. 
Figure 3 A illustrates an embodiment of a QWERTY keyboard of the hand-held 
1 5 device in accordance with the present invention. 

Figure 3B illustrates an embodiment of a layout of application buttons on the 
hand-held device. 

Figure 4 illustrates an embodiment of the hand-held device comprising a 
handwriting area upon which a user can enter symbols through the use of a display data 
20 entry device such as a stylus or his or her finger. 

Figure 5 illustrates an event flow diagram of an embodiment of one or more 
actions for providing symbols in an alternate symbol state. 
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Figure 6A illustrates an example of a display of at least one alternate symbol 
associated with a base symbol. 

Figure 6B illustrates another example of a display of at least one alternate symbol 

associated with a base symbol 
5 Figure 6C illustrates a third example of a display of at least one alternate symbol 

associated with a base symbol 

Figure 6D illustrates a fourth example of a display of at least one alternate symbol 
associated with a base symbol 

1 o DETAILED DESCRIPTION 

Figure 1 A illustrates a front view of an embodiment of a hand-held computing 
device with a lid opened in an extended position in accordance with the present 
invention. The embodiment of a hand-held device 100 depicted in Figure 1 A comprises a 
case 101 having a front side 104, and sides, for example, a first side 102. The front 
15 sidel04 comprises a display 128, in this embodiment a liquid crystal display (LCD) that 
is a touch-sensitive display. For example, the touch-sensitive display may include a 
digitizer. A user may use his or her finger or a stylus for data entry, for example, by 
indicating selections from a menu. In one embodiment, the display is a color display that 
displays color images. The front side 104 also comprises a keyboard 103, e.g. a 
20 QWERTY keyboard. User input devices embodied as application buttons 1 12, 1 14, 1 16, 
1 1 8 and 120 are also shown. In this embodiment, one of the application buttons 1 12 is 
associated with a telephone application. Another application button 1 14 is associated 
with a calendar application. Another application button 1 16 is associated with a scroll- 
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up, scroll-down feature. Another application button 118 may launch an Internet access 
application. An application button may be associated with more than one function or 
more than one application. In another example, an application button 120 may be 
associated with a messaging service, examples of which are electronic mail, e-mail, and 
5 Short Messaging Service (SMS). 

Attached to the front side 104 above the display 128 is a lid 106. In this 
embodiment, the lid 106 is a flip lid that is hinged 144 above the display. The lid 
includes a transparent portion 108 through which a user can see the display even if the lid 
is closed. The lid 106 further has a speaker 1 10 in this embodiment. 
1 0 As illustrated in this embodiment, the first side 1 02 includes a user input device, 

in this embodiment, a jog rocker 126. Also included on the side 102 of the device is a 
plug-in socket 130 for a hands-free speaker attachment. Examples of a hands-free 
speaker attachment may include a full headset or a single earpiece. 

Figure IB illustrates a back side 142 of the embodiment of the hand-held device 
1 5 of Figure 1 A. Also shown is a top part 146 having an antenna 1 36, and a ringer switch 
132 which interact with a radio module (See Figure 2 234) inside of the case that 
provides for radio communications including cellular telephone functionality. In this 
embodiment, a power switch 138 is shown on the upper portion of the device. In this 
embodiment, a two-color light emitting diode (LED) 152 is shown on the upper portion 
20 of the device. In this embodiment, an infrared panel port 1 54 is shown on the upper 
portion of the device. The top part of the device includes an opening of a stylus holder 
134. The stylus holder 134 extends cylindrically down the back side of the device. In 
this illustration of an embodiment of the device, a stylus is seated in the device as 
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illustrated by a head 804 of the stylus. In addition, the back side 142 is shown in this 
embodiment to comprise a door 140. The door holds an identification card. One 
example of such an identification card is a subscriber identification module (SIM) card 
typically used with portable telephones used within the Global System for Mobile (GSM) 
5 communications networks. This feature allows the user to place the card in another 
device. For example, if a user is on another continent, the user can slip the card into 
another voice communication device such as a telephone designed to work at the 
frequencies allocated for that continent, country or group of countries. 

Figure 1C illustrates a view of the front side of the embodiment of the hand-held 
10 device of Figure 1 A in which the lid 106 is in a closed position. The lid extends over the 
keyboard and the display in the closed position. The lid having a transparent portion 108 
provides a convenience to the customer and protection to the display of the device at the 
same time. For example, an alert of an incoming call notification is displayed on the 
display which is visible through the transparent lid. In this way, the lid also provides a 
1 5 protective cover of the device's display. 

The case 101 encloses logic, for example hardware and / or software, for 
providing communication services and logic for symbol processing. Examples of 
symbols are letters, numbers, punctuation marks, emoticons typically used in e-mail 
messages, shortcut function keys, and mathematical symbols. An example of 
20 communication services is voice communications. Figure 2 illustrates a block diagram of 
an embodiment of a system 200 comprising units making up the hardware and / or 
software for performing these functions. The system may be implemented in a printed 
circuit board. The system comprises a microprocessor 202, a read only memory (ROM) 
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204, a synchronous dynamic random access memory (SDRAM) 206, a user interface unit 
208 for processing input from the keyboard 103 or a handwriting area (see Figure 7, 702), 
the jog rocker 126, the ringer switch 132 and the power switch 138. Also this unit 208 
detects the state of the lid position to determine if the lid is closed or open. Connected to 
the microprocessor as well is charging circuitry 210 for providing power from the battery 
212. In one embodiment, the microprocessor is a Dragonball VZ MC68VZ328 33 MHz 
processor. In one embodiment, the battery is a Li-Ion 600 mAH battery. The charging 
circuitry is also connected to a cradle connector 226. The microprocessor is also 
connected with a liquid crystal display unit 214, a touch panel unit 216 and a backlight 
unit 218. The microprocessor 202 is also connected to a peripheral device transceiver 
224, an example of which is a Universal Serial Bus (USB) transceiver, and an 
input/output (I/O) port 222, an example of which is an EIA-232 or a RS-232 port, both of 
which are also connected to the cradle connector 226. The microprocessor 202 is also 
connected to an infrared communication transceiver 220, an example of which is an 
Infrared Data Association (IrDA) transceiver. The microprocessor is also connected with 
speaker unit 228, an example of which is a piezo speaker, a red/green light emitting diode 
(LED) 230 unit, and a vibrator unit 232. 

The microprocessor is also connected to a radio module 234 that provides for 
radio communications including cellular telephone functionality. An example of a radio 
module that may be used is a Wavecom Wismo GSM Module. Other services provided 
include Internet access and text messaging. An example of a text messaging service is 
the Short Message Service (SMS) which provides for sending and receiving short text 
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messages from the hand-held computing device to another communication device, for 
example a personal digital assistant or a mobile phone. 

The radio module is connected to an antenna 236, an amplifier 238 which is also 
connected to an internal speaker 240. The radio module is also connected to an internal 
microphone 242 as well as a card detector unit 244 and a hands-free attachment 
connector 246. 

Those of skill in the art will understand that logic, for example hardware and / or 
software, enclosed within the device, for example the system depicted in Figure 2, 
processes input from user input devices in different forms and performs functions with 
respect to the input. For example, the user interface unit 208 may be embodied in 
hardware and/or software under the control of the microprocessor 202. The 
microprocessor 202 may execute software instructions for receiving input and responding 
to it appropriately from software application programs 205 stored in a memory such as 
read only memory 204, and / or random access memory such as the illustrated SDRAM 
206, and/or in the user interface unit 208. 

A keyboard, such as a QWERTY keyboard, is advantageous for e-mail and other 
forms of electronic messaging in a hand-held device because data entry using a keyboard 
is more accurate than data entry using handwriting recognition software. 

In one version of the invention, a version of a QWERTY keyboard is used. For 
example, Figure 3 A illustrates an embodiment of a keyboard 103 that may be part of a 
hand-held computing device. The keyboard comprises the following keys: the letters a-z 
(26 keys), punctuation marks such as a period "." key. In one embodiment, the . " 
symbol key 630 is a dedicated key for processing related to a request for an alternate 
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symbol linked to abase symbol. Keys are also provided for such functions as "Space", 
"Return", "Backspace", as well as a "Shift" key, an "Option" key, and a 
"Menu/Command" key. It will be noted that beginning in the top row at the left side, the 
first six letters from left to right are "qwerty". The keys may be hardware keys or part of 

5 a displayed keyboard. 

This embodiment includes an option key 628. Responsive to receiving input 
indicating that an option state is in effect, an option symbol associated with a key will be 
displayed responsive to receiving input indicating activation of the key. As shown, an 
option symbol associated with a key is displayed on the key. For example, as shown, 

10 normal symbol keys "y", "u" and "i" have respectively as their option symbols "1", "2" 
and "3". 

Figure 3B illustrates an embodiment of the layout of the application buttons of the 
hand-held device. In this embodiment, one of the application buttons 632 is associated 
with a telephone application. Another application button 634 is associated with a 

1 5 calendar application. Another application button 636 is associated with a scroll-up, 

scroll-down feature. For example, a user may use this feature to scroll through entries in 
a menu. In this embodiment, application button 638 is associated with an application that 
provides Internet access. In this embodiment, an application button 640 is associated 
with a messaging application. 

20 i Figure 4 illustrates another version of the embodiment of the hand- 

held device of Figures 1A, IB and 1C. In the version of Figure 4, the device comprises a 
handwriting area 702 upon which a user can enter symbols through the use of a stylus or 
his or her finger or other display contact data entry device. The icons surrounding the 
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handwriting area are those typically used with the Graffiti™ program typically 
implemented in PALM OS® devices. The icons when tapped perform a function 
associated with the icon. For example, an applications icon 708 opens applications when 
tapped. The other icons are the menus icon 710, the calculator icon 706, and the find 
5 function icon 704 which allows a user to find text anywhere in his data. In the lid-closed 
position of the device, the embodiment of Figure 4 would appear as the embodiment of 
the device having a transparent lid 108 in the closed position as illustrated in Figure 1C. 
This embodiment of a hand-held device also comprises a system such as the example 
illustrated in Figure 2 comprising logic, for example hardware and / or software, for 
10 providing communication services and symbol processing. For example, this device also 
includes radio communications functionality, for example, cellular telephone 
functionality as discussed above. 

The number of symbols that can be displayed on a keyboard including displayed 
keyboards cannot typically cover every symbol. For example, a symbol such as the letter 
1 5 "e" is not typically available in a plurality of accented versions on a keyboard layout for 
the English language. Besides desiring to include in a message a different version of a 
symbol, a user may desire to find a symbol of the same type as one available on a 
keyboard. For example, a user may need the symbol for the Japanese yen, a symbol of 
the same type as the U.S. dollar sign "$" typically available on keyboards or accepted by 
20 handwriting recognition software. Furthermore, a user may desire to use a symbol 
having a relationship or connection with a symbol he has typed. Examples of 
relationships or connections may be logical relationships, relationships by association, or 
user-defined relationships. For example, the trademark symbol, ™ may be an alternate 
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symbol linked to the letter "T". One association is that the symbol visually includes the 
letter. Another association is that the name of the symbol commonly used, e.g. 
"trademark symbol" begins with the base symbol In another example, a command 
symbol that causes an action may also be an associated alternate symbol. An example of 
a command symbol is a <shortcut> symbol key, such as one that causes input to be 
received, the input indicating a command or request that is also associated with the 
activation of a sequence of keys or displayed items. Shortcut symbols may be associated 
with the letter "S" because shortcut begins with "S". Additionally, a shortcut symbol 
may be an alternate symbol associated with one or more of the base symbols represented 
by the keys or displayed items in the sequence. 

Another kind of association or connection based relationship is a visual 
association relationship. For example, the British pound symbol "£" may also be 
associated with the letter "L" because they look similar. They may be visually associated 
because the both have a vertical line at the bottom of which another line protrudes in a 
general horizontal direction. 

The relationship may also be context related. For example, the calendar 
application software may provide a list of alternate symbols including alternate symbols 
appropriate for the calendar application. For example, for the letter "j", short symbol 
sequences of the words "June" and "July" may be provided in the list. 

A base symbol is a symbol to which at least one alternate symbol is linked. A 
base symbol commonly appears on a key, or is displayed in an initial view of a display 
for text entry or is a handwritten symbol recognized by handwriting entry software. 
Examples of alternate symbols are accented characters, punctuation marks that do not 
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appear on a keyboard, and short symbol sequences. An example of a common short 
symbol sequence is an emoticon used in e-mail messages to convey tone or feelings. 

Figure 5 illustrates an event flow diagram of an embodiment of one or more 
actions for providing alternate symbols associated or linked with a base symbol. In one 
embodiment, the one or more actions may be implemented by the microprocessor 202 
executing software instructions causing the actions described. Those of skill in the art 
will recognize that one or more of the methods may be implemented in embodiments of 
hardware and / or software and combinations thereof. For example, instructions for 
performing the one or more actions may be embodied within a computer usable medium. 

Upon the occurrence of the event of receiving 804 input indicating a request for 
one or more alternate symbols linked to a base symbol, at least one alternate symbol for 
is displayed 808 for the base symbol. The order in which the alternate symbols are 
displayed may be based upon a criteria. A user may request an alternate symbol display 
view by typing a key sequence, for example <optionXspace> after entering a base 
symbol. Also, a sequence of contacts with a stylus or finger on a screen icon or a portion 
of the screen such as that displaying the base symbol may be detected to indicate a 
request. Requests may also be indicated by pressing a key dedicated for processing 
alternate symbols such as the ". . " key 630 displayed on the keyboard in Figure 3 A. Or 
the user may request an alternate symbol for a base symbol by holding a key representing 
the base symbol down or maintaining a display contact for a designated amount of time. 

Each of figures 6A, 6B and 6C illustrate an example of a view of at least one 
alternate symbol associated with a base symbol. In each of these examples, the view is 
embodied as a pop-up menu displaying a list of alternate symbols. Figure 6 A illustrates a 
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pop-up menu of a plurality of accented versions of the base letter "e". In the example of 
Figure 6A, the base symbol "e" appears at the top of the pop-up menu followed by a list 
of alternate symbols associated with the base symbol. Figure 6B illustrates another 
version of a pop-up menu of symbols of the same type as the "$". For example, the 
symbol "£" for the British pound and the symbol "¥" for the Japanese yen are also 
displayed. The current alternate symbol available for selection is indicated in this 
example by highlighting. Figure 6C illustrates a pop-up menu display of examples of 
alternate symbols that have a logical or associational connection with the base symbol of 
colon ":" that the user has typed. The semi-colon ";" and the emoticons ":-)" indicating a 
smile or happiness, ":-(" indicating sadness, and the ";-)" typically associated with a 
smile with a wink or tongue in cheek expression. The semi-colon is typically associated 
with a colon either by sharing a common syllable or being on the same key in typical 
desktop keyboard. By association, the tongue in cheek emoticon uses a semi-colon so it 
is also linked or associated with the colon. The other displayed emoticons have a colon 
as part of their short sequences of symbols so they are logically or associationally 
connected to the colon. 

In another embodiment illustrated in Figure 6D, the display is updated showing 
the base symbol replaced by a current alternate symbol. Responsive to receiving 805 
input indicating a request for another alternate symbol to be the current alternate symbol 
for the base symbol, the display is updated 81 1 to indicate another current alternate 
symbol for the base symbol. The user may request another alternate symbol to be the 
current alternate symbol by pressing the dedicated key, or using the scroll button or the 
jog rocker scroll functionality. The alternate symbol may be indicated as the current 
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alternate symbol that may be selected for the base symbol by highlighting it in the pop-up 
list or by replacing the base symbol with it on the screen. 

Similarly, in another embodiment, a base symbol may have an alternate symbol 
such as a sequence of symbols or other symbols that are added to the base symbol. For 
5 example, the alternate symbols linked with the base symbol "." may include "com", 
"org", "biz", or "net" which are added after the period. The association of the period 
with these short symbol sequences is their common use in the same website name. The 
period is often called "dot" in this context. In natural language also, the "period" has 
become associated with the phrase "dot.com" to refer to a business type. Alternatively, 
1 0 the alternate symbols can include the symbol so that the symbol is replaced by the 

complete sequence. An example of this is the smile or happy emoticon linked to the base 
symbol, the colon. 

Upon the occurrence of the event of receiving 820 input indicating selection of an 
alternate symbol for the base symbol, the base symbol is replaced 822 with the selected 

1 5 alternate symbol on the display. In one embodiment, a user may select an alternate 

symbol from a display by typing another symbol so that the alternate symbol is displayed 
and is followed by this other symbol. In other embodiments, the user may select an 
alternate symbol by using a specific key sequence or tapping the display either anywhere 
or in a specific location, for example with a finger or stylus. In other embodiments, 

20 examples of other keys that may be used to indicate selection are a <return> key, an 
<enter> key, or a <space> key. Alternatively, activating the jog rocker, for example by 
pressing it, may be used. 
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Upon the occurrence of receiving 810 input indicating a request to exit from the 
alternate symbol state, the display is updated 812 to display the base symbol. In one 
embodiment, the user may select the alternate symbol and type another symbol such as 
the next character in the sentence in order to exit the alternate symbol state. Also other 
examples of keys or key sequences that may be used to select an alternate symbol and 
exit the alternate symbol state include the <return> key, the <space> key, the jog rocker, 
or a tap on the desired alternate symbol displayed on the touch-sensitive display. 
However, a user may desire to leave the alternate symbol state without selecting an 
alternate symbol. In one embodiment, receiving input indicating activation of the 
<backspace> key causes an exit from the alternate symbol state. 

In the event of receiving 850 input indicating a request for an alternate symbol for 
a base symbol not linked to any alternate symbol, an alert is displayed 852. Alerting the 
user may be performed in one way by displaying a message to the user. In another way, 
the user may be alerted by an audio sound such as a beep. 

A user may not find the alternate symbol she was looking for in the list linked to 
the current base symbol. The user may desire to browse lists of alternate symbols 
associated with other base symbols. Responsive to the event of receiving 840 input 
indicating selection of a different base symbol while in the alternate symbol state, a view, 
such as a pop-up menu, of alternate symbols for the different base symbol is displayed 
842. 

The order of the list of one or more alternate symbols associated with a base 
symbol may be displayed based on different criteria. Information associated with the list 
may be stored with the list indicating a relationship to the criteria. For example, the 
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information may be embodied as a header associated with the list. For example, one 
example of criteria is that the most common alternate symbol based on statistics may be 
displayed first. Such statistics may be pre-determined statistics based on general usage or 
surveys of all users. Alternatively, the order may be based on prior usage or context. For 
5 example, for each alternate symbol in a list, a frequency selection indicator may be 
associated with an alternate symbol to indicate the frequency with which that alternate 
symbol is selected. An example of a frequency selection indicator is a count of the 
number of selection times for a session or over the lifetime of the device. The alternate 
symbol with the highest frequency count is displayed first. Alternatively, the last 
1 0 selected alternate symbol may be the criteria for determining which alternate symbol is 
displayed first for a base symbol. The content of the list may also be configured based on 
these suggested criteria and others. For example, the list may be localized for a particular 
language. For example, devices for French language usage may have different symbols or 
a different order than those set-up for German language usage. Of course, either or both 
15 of the order and content may also be user configurable in other embodiments. 

The above description is included to illustrate the operation of one or more 
embodiments and is not meant to limit the scope of the invention. The scope of the 
invention is to be limited only by the following claims. From the above discussion, many 
variations will be apparent to those skilled in the art that would yet be encompassed by 
20 the spirit and scope of the invention. 
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